durability: retry SMB replace-existing; triage derives PC when the me… #2
+292
−3
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
…rge's write was lost (v0.8.9) Writing ALTO to \wl.mydata.purdue.edu, 3 of 41 pages in one object failed the confidence merge with WinError 5 while pages written seconds either side succeeded. Not a permissions problem: the files are not read-only, no temp file leaked, and a real ACL denial would have failed all 41. It is the familiar Windows/SMB case where a transient handle on the destination — antivirus, the Search indexer, an oplock break in flight — makes replace-existing fail, and Windows reports it as ERROR_ACCESS_DENIED (5) as readily as ERROR_SHARING_VIOLATION (32). Errno cannot tell transient from real; only retrying can. atomic_write_bytes now routes the final rename through _replace_with_retry: 6 attempts with backoff (~1.5s) on winerror 5/32 only. A genuine denial still raises, just later. Disk-full and POSIX-style errors fail fast, unretried. The second half of the bug was how the failure presented. Tesseract's own ALTO renderer emits per-word WC natively (verified against 5.5.0); our merge re-derives those and adds Page/@PC. So a lost merge write keeps every word confidence and drops only the page mean — and review_pages filtered on `p.pc is not None`, so those pages vanished from the correction queue entirely. On an object with mean WC 0.351 that is the worst possible page to hide. triage.scan_alto_file now derives PC from the word WCs when the attribute is absent — the same mean the merge would have stored, and the derivation remediate already applies when it rewrites PC after an edit. A stored PC is never overridden, and unscored (GLMOCR) and empty pages still report None. Measured on the affected object: review queue 29 -> 32 pages. The Review tab reads the same scan, so its worst-first ordering recovers too. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
schipp0
added a commit
that referenced
this pull request
Aug 6, 2026
…vation land (v0.8.13) The remote main gained PRs #1–#2 (branch preprocessing-binarize-default) while local main re-landed the binarize-off default independently and advanced through v0.8.12 — the histories diverged. Resolutions: - KEPT from the remote line (was missing locally): atomic_write_bytes' _replace_with_retry (6 attempts/backoff on WinError 5/32 — transient SMB handle failures during replace-existing), triage deriving Page PC from word WCs when the merge's write was lost, both test files, and CLAUDE.md's --psm 1 rotated-scans section. - KEPT from local: everything v0.8.8–v0.8.12 (add-word, clear-page, Repackage button, reading-order mode, audit refactors); version resolved to 0.8.13 since the merged tree adds behavior over 0.8.12. - RECONCILED by hand: git auto-merge had silently kept BOTH _migrate_settings definitions in gui.py (the second shadowed the first). Now one function honoring BOTH marker names — the remote build's "binarize_default_2026_07_30" and the local "_migrated_binarize_default_off" — so a settings file migrated by either build is never re-migrated (which would clear a deliberate binarize re-opt-in). Remote's gentler semantics kept: an explicit pre-migration do_binarize:false survives; clearing a stale true logs a user-facing note. test_preprocess_defaults.py is the union of both versions plus a new legacy-marker regression test. Suite: 184 tests green. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Sign in
to join this conversation on GitHub.
1 participant
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
…rge's write was lost (v0.8.9)
Writing ALTO to \wl.mydata.purdue.edu, 3 of 41 pages in one object failed the confidence merge with WinError 5 while pages written seconds either side succeeded. Not a permissions problem: the files are not read-only, no temp file leaked, and a real ACL denial would have failed all 41. It is the familiar Windows/SMB case where a transient handle on the destination — antivirus, the Search indexer, an oplock break in flight — makes replace-existing fail, and Windows reports it as ERROR_ACCESS_DENIED (5) as readily as ERROR_SHARING_VIOLATION (32). Errno cannot tell transient from real; only retrying can.
atomic_write_bytes now routes the final rename through _replace_with_retry: 6 attempts with backoff (~1.5s) on winerror 5/32 only. A genuine denial still raises, just later. Disk-full and POSIX-style errors fail fast, unretried.
The second half of the bug was how the failure presented. Tesseract's own ALTO renderer emits per-word WC natively (verified against 5.5.0); our merge re-derives those and adds Page/@PC. So a lost merge write keeps every word confidence and drops only the page mean — and review_pages filtered on
p.pc is not None, so those pages vanished from the correction queue entirely. On an object with mean WC 0.351 that is the worst possible page to hide.triage.scan_alto_file now derives PC from the word WCs when the attribute is absent — the same mean the merge would have stored, and the derivation remediate already applies when it rewrites PC after an edit. A stored PC is never overridden, and unscored (GLMOCR) and empty pages still report None. Measured on the affected object: review queue 29 -> 32 pages. The Review tab reads the same scan, so its worst-first ordering recovers too.